Focus detection apparatus and control method therefor

ABSTRACT

A focus detection apparatus includes an image pickup unit configured to perform photoelectric conversion on a luminous flux having passed through an imaging optical system, a focus detection unit configured to detect a focusing state based on a signal generated by the image pickup unit, a setting unit configured to set a first region within an image generated by the image pickup unit, a display controller configured to control such that an index representing the focusing state detected by the focus detection unit within the first region can be superimposed on the image, and an obtaining unit configured to obtain information regarding a predetermined subject in the image. In this case, the setting unit sets a position of the first region based on information regarding at least one of a position, a size, and a direction of the predetermined subject in the image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a focus detection apparatus including aconfiguration for detecting a focusing state and a configuration forcontrolling an indication of a focusing state.

2. Description of the Related Art

With a high definition camcorder supporting full high definition or 4Kvideo capture, it is not easy for a photographer to implement precisefocus adjustment by performing a manual focus operation (hereinafter,called an MF operation) on a subject. Particularly when a focusadjustment is performed by checking focusing on a viewfinder or a panel,for example, defocusing may occur which cannot be found on theviewfinder or panel, for example.

In order to solve this problem, an apparatus has been known which has afunction for displaying an enlarged image of a part of the field of viewon a liquid crystal display. Japanese Patent Laid-Open No. 2012-186670proposes a method for identifying the position of a feature partincluded in a subject image through an image analysis process anddisplaying an enlarged image of the feature part at the identifiedposition on a liquid crystal display in a case where a criterion forenlarged display is not registered. More specifically, in a case whereone human figure is a subject, images of both eyes of the human figureare enlarged and are displayed simultaneously. In a case where aplurality of human figures are subjects, images of face parts of all ofthe human figures are enlarged and are displayed simultaneously.

However, according to Japanese Patent Laid-Open No. 2012-186670, thein-focus direction is not recognized from the enlarged image, andbecause an MF operation is performed by checking the enlarged image,there may be a possibility that the focus lens may be moved by passingthrough the in-focus position unintentionally.

Japanese Patent Laid-Open No. 2007-279334 discloses a configurationincluding a main imaging part and sub-imaging parts for front focusevaluation and back focus evaluation, wherein an in-focus state markerindicative of the in-focus direction is displayed as a result ofcalculation of the in focus levels of the front focus state and the backfocus state. Japanese Patent Laid-Open No. 2007-248615 proposes a methodfor displaying a bar indicative of a degree of in-focus as a result ofcalculation of an in-focus evaluation value during an MF operation.Japanese Patent Laid-Open No. 2005-140943 proposes a focus assist methodin an imaging apparatus for displaying a plurality of indicatorsindicative of a change in focusing state caused by a move of a focuslens. Japanese Patent Laid-Open No. 2009-122593 proposes a method forchanging a display period of an indicator of a focusing state inaccordance with the operation performed on a focus lens.

Displaying the in-focus state marker disclosed in Japanese PatentLaid-Open No. 2007-279334 for each of all feature portions as inJapanese Patent Laid-Open No. 2012-186670 may complicate the resultingimage as well as an increase in calculation load. Accordingly, selectionof an image region for displaying an in-focus state marker may beexpected. However, Japanese Patent Laid-Open No. 2012-186670 does notdisclose how an image region for displaying an in-focus state marker isto be set in accordance with a subject. It does not disclose a methodfor changing the image region for displaying an in-focus state marker inaccordance with a user operation.

SUMMARY OF THE INVENTION

The present invention provides a function of indicating a focusingstate, wherein the indication of the focusing state can be displayed ona proper image region according to the state of a subject.

According to an aspect of the present invention, there is provided afocus detection apparatus including an image pickup unit configured toperform photoelectric conversion on a luminous flux having passedthrough an imaging optical system, a focus detection unit configured todetect a focusing state based on a signal generated by the image pickupunit, a setting unit configured to set a first region within an imagegenerated by the image pickup unit, a display controller configured tocontrol such that an index representing the focusing state detected bythe focus detection unit within the first region can be superimposed onthe image, and an obtaining unit configured to obtain informationregarding a predetermined subject in the image, wherein the setting unitsets a position of the first region based on information regarding atleast one of a position, a size, and a direction of the predeterminedsubject in the image.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating configurations of a camera and alens unit according to an exemplary embodiment.

FIGS. 2A and 2B illustrate pixel configurations according to anexemplary embodiment.

FIG. 3 is a flowchart illustrating processing for controlling display ofa focus assist frame according to an exemplary embodiment.

FIGS. 4A to 4E illustrate a focus detection region according to anexemplary embodiment.

FIGS. 5A to 5C illustrate forms of focus assist display according to anexemplary embodiment.

FIG. 6 is a flowchart illustrating focus detection processing accordingto an exemplary embodiment.

FIGS. 7A to 7C illustrate image signals obtained from focus detectionregions according to an exemplary embodiment.

FIGS. 8A to 8D illustrate a correlation calculation method according toan exemplary embodiment.

FIG. 9 is a flowchart illustrating processing for setting a focus assistdisplay region according to a first exemplary embodiment.

FIGS. 10A to 10G illustrate examples of a displayed focus assist frameaccording to the first exemplary embodiment.

FIG. 11 is a flowchart illustrating processing for setting a focusassist display region according to a second exemplary embodiment.

FIGS. 12A and 12B illustrates examples of a displayed focus assist frameaccording to the second exemplary embodiment.

FIG. 13 is a flowchart illustrating focus assist display region changingprocessing according to an exemplary embodiment.

FIGS. 14A to 14C illustrate examples of focus assist displays accordingto an exemplary embodiment.

FIG. 15 illustrates a relationship between moving amount of a focusdetection region and focus assist display according to an exemplaryembodiment.

FIGS. 16A and 16B are flowcharts illustrating a main flow of controlover display of focus assist display according to an exemplaryembodiment.

FIG. 17 is a flowchart illustrating an example of processing for settinga focus detection region according to an exemplary embodiment.

FIG. 18 is a flowchart illustrating processing for averaging focusdetection results according to an exemplary embodiment.

FIG. 19 is a flowchart illustrating processing for calculating an angleindex according to an exemplary embodiment.

FIG. 20 is a flowchart illustrating processing for calculating adirection index according to an exemplary embodiment.

FIGS. 21A and 21B illustrate examples of an additionally set focusdetection region for detecting a subject according to an exemplaryembodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described belowin detail with reference to the attached drawings. The followingembodiments are given only for purpose of illustration of implementationof the present invention, and it should be understood that the presentinvention is limited to the following embodiments.

First Exemplary Embodiment

FIG. 1 is a block diagram illustrating a configuration of aninterchangeable-lens camera including a lens unit and a camera main bodyaccording to a first exemplary embodiment. As illustrated in FIG. 1, acamera system according to this exemplary embodiment includes a lensunit 10 and a camera main body 20. A lens control unit 106 configured togenerally control operations of the whole lens unit 10 and a cameracontrol unit 207 configured to generally control the whole cameracommunicate data with each other. According to this exemplaryembodiment, an interchangeable-lens camera will be described as anexample. However, the present invention is applicable to a cameraintegrated with a lens.

A configuration of the lens unit 10 will be described first. The lensunit 10 has an imaging optical system including a fixed lens 101, aniris 102, a focus lens 103, and a zoom lens (not illustrated). The iris102 is driven by an iris drive unit 104 and is configured to control thequantity of light incident on an image sensing element 201, which willbe described below. The focus lens 103 is driven by a focus lens driveunit 105 and is configured to perform focus adjustment. The zoom lens,not illustrated, is driven by a zoom lens drive unit to performadjustment of zooming. According to this exemplary embodiment, the zoomlens and the zoom lens drive unit are not required components.

The iris drive unit 104, focus lens drive unit 105, and zoom lens driveunit are controlled by a lens control unit 106, and the open area amountof the iris 102 and the positions of the focus lens 103 and zoom lensare determined. When a user performs an operation of focusing or zoomingthrough the lens operating unit 107, the lens control unit 106 executesa control based on the user operation. According to this exemplaryembodiment, a user may move the focus lens 103 through the lensoperating unit 107 so that manual focus adjustment operation(hereinafter, called an MF operation) can be performed.

The lens control unit 106 controls the iris drive unit 104, the focuslens drive unit 105, and the zoom lens drive unit in accordance withcontrol instructions and control information received from a cameracontrol unit 207, which will be described below, and transmits lensinformation to the camera control unit 207.

Next, a configuration of the camera main body 20 including a focusdetection apparatus according to this exemplary embodiment will bedescribed. The camera main body 20 is configured to be capable ofgenerating an image pickup signal from a luminous flux having passedthrough the imaging optical system in the lens unit 10. The imagesensing element 201 may be a CCD or CMOS sensor. A luminous flux havingpassed through the imaging optical system focuses on a light receivingsurface of the image sensing element 201, and the formed subject imageis converted to electric charges corresponding to the quantity ofincident light by photodiodes (photoelectric conversion). The electriccharges accumulated in the photodiodes are sequentially read from theimage sensing element 201 as a voltage signal corresponding to theelectric charges based on a driving pulse given from a timing generator209 in accordance with a command from the camera control unit 207.

An image sensing element which does not support image plane phasedifference detection for focus detection may have a Bayer pattern pixelconfiguration as illustrated in FIG. 2A, for example. On the other hand,the image sensing element 201 according to this exemplary embodimenthold a plurality of (two in this exemplary embodiment) photodiodes inone pixel as illustrated in FIG. 2B in order to implement image planephase difference detection for focus detection. A luminous flux isdemultiplexed by the microlens and is focused by the two photodiodes sothat two signals for image pickup and focus detection can be obtained. Asignal (A+B) acquired by adding the signals of the two photodiodes is animage pickup signal, and the signals (A, B) of the photodiodes are twoimage signals for focus detection. According to this exemplaryembodiment, the method for obtaining the two image signals is notlimited to the configuration for reading out the two image signals. Forexample, in consideration of a processing load, the added signal (A+B)and one image signal (such as A) may be read out, and the other imagesignal (such as B) may be acquired from a difference between the addedsignal (A+B) and the one image signal (such as A). A focus detectionsignal processing unit 204, which will be described below, performscorrelation computation on the two image signals for focus detection tocalculate image defocus amount and reliability information.

According to this exemplary embodiment, one pixel has two photodiodes.However, the number of photodiodes is not limited to two, but morephotodiodes may be provided therein. A plurality of pixels havinglight-receiving sections having apertures at different positions withrespect to a microlens. In other words, two signals, such as an A imagesignal and a B image signal, for detection of a phase difference mayonly be required as a result. An image sensing element supporting focusdetection based on image plane phase difference detection may include apixel for focus detection, without limiting to the configuration inwhich a plurality of photodiodes are provided in one pixel as in thisexemplary embodiment.

The image pickup signal and the signal for focus detection (hereinafter,also called a focus detection signal) read out from the image sensingelement 201 are input to a CDS (correlated double sampling)/AGC(automatic gain control) converter 202 and undergo correlated doublesampling for removal of reset noise, gain adjustment, and signaldigitization. The CDS/AGC converter 202 outputs the image pickup signalto a camera signal processing unit 203 and a subject detecting unit 210and the signal for focus detection based on image plane phase differencedetection to the focus detection signal processing unit 204.

The camera signal processing unit 203 transmits the image pickup signaloutput from the CDS/AGC converter 202 to a display unit 205. The displayunit 205 is a display device which may be configured by a liquid crystaldisplay (LCD) or an organic electroluminescence (EL) display, forexample, and displays an image based on an image pickup signal. In amode for recording an image pickup signal, the image pickup signal isrecorded in a recording unit 206.

The focus detection signal processing unit 204 performs correlationcomputation based on two image signals for focus detection output fromthe CDS/AGC converter 202 to calculate an image defocus amount andreliability information (such as a matching degree of two images, asteeping degree of two images, contrast information, saturationinformation, and defect information). The calculated image defocusamount and reliability information are output to the camera control unit207. Details of the correlation computation will be described below withreference to FIGS. 7A to 7C and FIGS. 8A to 8D.

This exemplary embodiment is configured such that a focus assistindicator is displayed on the display unit 205 in an MF mode in whichfocus adjustment is implemented by performing an MF operation. The focusassist indicator represents a focusing state within a focus assist framesuperimposed on an image. The focusing state within the focus assistframe is detected based on a result of the correlation computationperformed by the focus detection signal processing unit 204. A user canvisually recognize information regarding a defocus amount and a defocusdirection from the focus assist display to perform an MF operationintended by the user. Details of the focus assist display will bedescribed below. On the other hand, in an AF mode in which focusadjustment is implemented automatically, focus adjustment is implementedby using a result of a correlation computation performed by the focusdetection signal processing unit 204. In the AF mode, an AF frameindicating a region for obtaining a signal for performing focusadjustment is superimposed on an image, and focus assist display is notperformed.

The camera control unit 207 performs controls by exchanging informationwith components within the camera main body 20. The camera control unit207 may not only perform processing within the camera main body 20 butalso implement a camera function operated by a user in accordance withan input from the camera operating unit 208, such as powering ON/OFF, asetting change, a recording start, start of focus detection control,checking a recorded image, and selection of a focus detection frame. Asdescribed above, the camera control unit 207 exchanges information withthe lens control unit 106 within the lens unit 10 and transmits acontrol instruction and control information for the imaging opticalsystem and obtains information regarding the internal configuration ofthe lens unit.

The subject detecting unit 210 is a block related to detection such as aface detection and a human body detection, and performs a publicly knowndetection process on an image pickup signal output from the CDS/AGCconverter 202 and detects a specific subject region within an imagingscreen. In other words, the subject detecting unit 210 is configured todetect a predetermined subject from an image pickup signal. A publiclyknown method may be used for the face detection, and detail descriptionswill be omitted.

Focus Assist Frame Display Control as a Whole

Next, a sequence of a whole focus assist display control to be performedby the camera control unit 207 will be described with reference to FIG.3. The processing illustrated in FIG. 3 is executed periodically basedon an operation cycle of the camera.

First of all, in step S301, the camera control unit 207 instructs thesubject detecting unit 210 to detect a specific subject region within animaging screen. An example in which a region of the face of a humanfigure is set as the specific subject region will be described accordingto this exemplary embodiment.

Next, in step S302, the camera control unit 207 performs processing forsetting a region for focus assist display (hereinafter, called a focusassist display region or a focus assist frame). The focus assist displayregion corresponds to a range (focus detection range) in whichcorrelation computation is performed by the focus detection signalprocessing unit 204 in focus detection processing, which will bedescribed below. In other words, the processing for setting a focusassist display region can also be processing for setting the focusdetection range. Details of the processing for setting a focus assistdisplay region will be described below with reference to FIG. 9.

Next, in step S303, the camera control unit 207 performs processing forchanging the position of the focus assist frame in accordance with anoperation performed by a user. Details of the processing for changingthe focus assist display region will be described below with referenceto FIG. 13.

Next, in step S304, the focus detection signal processing unit 204performs focus detection processing in the focus detection rangecorresponding to the focus assist frame set in step S302. Details of thefocus detection processing will be described below with reference toFIG. 6.

Next, in step S305, the camera control unit 207 performs processing fordisplaying the focus assist frame on the display unit 205 and stops thefocus assist display control.

According to this exemplary embodiment, the focus assist display controlis executable in the MF mode and is not executed in the AF mode.

Processing for Setting Focus Detection Region

Next, processing for setting a region on which the focus detectionprocessing is performed in step S304 will be described with reference toFIGS. 4A to 4E. FIGS. 4A to 4E illustrate an example of a region forobtaining an image signal to be used for the focus detection processing.

FIG. 4A illustrates a focus detection range 402 on a pixel array 401.The region 404 necessary for implementing correlation computationincludes the focus detection range 402 and a shift region 403 necessaryfor correlation computation. FIG. 4A illustrates coordinates p, q, s,and t in the x-axis direction. The region 404 is from the coordinate pto the coordinate q, and the focus detection range 402 is from thecoordinate s to the coordinate t.

FIG. 4B illustrates focus detection regions 405 to 409 acquired bydividing the focus detection range 402 into five. As an example, anamount of focus shift is calculated for each focus detection region forperforming a focus detection according to this exemplary embodiment.According to this exemplary embodiment, a focus detection result basedon a most reliable region from the plurality of divided focus detectionregions, and the amount of focus shift calculated from the region isused for focus assist display. It should be noted that the number anddirection of divisions of a focus detection range is not limitedthereto.

FIG. 4C illustrates a temporary focus detection region 410 connectingthe focus detection regions 405 to 409 in FIG. 4B. As an example of thisexemplary embodiment, an amount of focus shift calculated from theregion connecting focus detection regions may be used for focus assistdisplay.

In a camera supporting a subject detection function, a focus detectionregion 419 can be set at a position of the detected face 420, asillustrated in FIG. 4D. In this case, one or a plurality of focusdetection regions 419 are set for the detected face 420, and one validdefocus amount and one valid defocus direction are calculated from afocus detection result obtained from the set focus detection region. Thevalid defocus amount and/or the valid defocus direction may be used forfocus assist display.

In an imaging apparatus supporting a touch input, for example, a focusdetection region may be set freely in accordance with a designation by auser. As illustrated in FIG. 4E, a focus detection region may be set ata designated position 421.

The method for determining the position and the width of a focusdetection region is not limited to details described according to thisexemplary embodiment but may changed or modified without departing fromthe scope and spirit of the present invention.

Display Forms of Focus Assist Frame

Next, display forms of focus assist frame according to this exemplaryembodiment will be described with reference to FIGS. 5A to 5C. FIGS. 5Ato 5C illustrate exemplary focus assist display forms.

A focus assist frame 500 is displayed in a region corresponding to thefocus detection range 402 illustrated in FIGS. 4A to 4E. The focusassist frame and the focus detection range here are not necessarilymatched but may substantially be matched. Graphic representations 502 to507 are visually indicative of a defocus amount and a defocus directionand move along a broken line 501. The broken line 501 is not displayedon the display unit 205.

Changes in the focus assist display mode in accordance with the focusingstate will be described in detail with reference to FIGS. 5A to 5C.First, FIG. 5A illustrates a focusing state in a case where the focuslens is present on a minimum-object-distance side about a subject withinthe focus assist frame 500. In a case where the focus lens is present onthe minimum-object-distance side, the graphic representation 502arranged outside of the broken line 501 and the two graphicrepresentations 503 and 504 arranged inside the broken line 501 aredisplayed. With the graphic representation 502 stopped at an upperposition, the graphic representations 503 and 504 move along the brokenline 501 horizontally symmetrically about the centerline in accordancewith the defocus amount. The distances from the graphic representations503 and 504 to the graphic representation 502 increase as the defocusamount increases.

FIG. 5B illustrates a focusing state in a case where the focus lens ispresent on the infinite end side about the subject within the focusassist frame 500. In a case where the focus lens is present on theinfinite end, the graphic representation 507 arranged inside the brokenline 501 and the two graphic representations 505 and 506 arrangedoutside the broken line 501 are displayed. With the graphicrepresentation 507 stopped at an upper position, the graphicrepresentations 505 and 506 move along the broken line 501 horizontallysymmetrically about the centerline in accordance with the defocusamount. The distances from the graphic representations 505 and 506 tothe graphic representation 507 increases as the defocus amountincreases.

FIG. 5C illustrates a focusing state in a case where the focus lens ispresent at an in-focus position about the subject within the focusassist frame 500. The graphic representation 502 can represent a statein which the graphic representations 505 and 506 are overlapped, and thegraphic representation 507 can represent a state in which the graphicrepresentations 503 and 504 are overlapped. When the focus lens is atthe in-focus position, the graphic representation 502 and the graphicrepresentation 507 position most closely.

FIG. 14A illustrates another example of a focus assist display 1400indicative of a focusing state. A focus detection region 1401 representsa region for detecting a focusing state, and display parts 1402 to 1405represent detected focusing states. When focus assist display isimplemented, the display parts are superimposed on an image displayed onthe display unit 205.

The angle display part 1402 is an index indicative of a distance to anin-focus position (corresponding to a defocus amount), and the in-focusposition display part 1403 indicates the in-focus position. The angledisplay part 1402 moves along a round part 1404 in accordance with afocusing state and is displayed at the same position as the in-focusposition display part 1403 when an in-focus state is obtained. When anin-focus state is obtained, the display parts may be displayed in colorsdifferent from those in an out-of-focus state. The angle formed by theangle display part 1402 and the in-focus position display part 1403varies in accordance with the focus detection result (defocus amount)obtained from the focus detection region 1401. The direction displaypart 1405 is an index indicative of the direction toward an in-focusstate. This indicates whether the out-of-focus state occurs toward theminimum-object-distance side or toward the infinite distance side aboutan in-focus position. As described above, the focusing state display isimplemented by using display parts indicative of the distance to anin-focus position, the direction, and an in-focus state.

The focus assist display 1400 configured as described above changes itsstate as illustrated in FIG. 14B in order to clearly notify a userwhether an in-focus state has been obtained or not. First of all,different angles are formed by the in-focus position display part 1403and the angle display part 1402 between a case where the focus lens 103is at a position away from an in-focus position (large blur) and a casewhere the focus lens 103 is a position close to the in-focus position(small blur). In other words, as illustrated in FIG. 14B, a larger angleθ1 for large blur than an angle θ2 for small blur can notify a user of adistance from the in-focus position. The angle changes smoothly inaccordance with the focusing state. At an in-focus state, the angledisplay part 1402 representing the angle and the in-focus positiondisplay part 1403 are superimposed. The display color or thickness ofthe round part 1404 may be changed at an in-focus state.

When the distance (defocus amount) to an in-focus position is notavailable and the in-focus direction is only available, the directiondisplay part 1405 is displayed without displaying the angle display part1402. When both of the distance and direction to an in-focus positionare not available, both of the angle display part 1402 and the directiondisplay part 1405 are not displayed, indicating that the focus detectionis invalid. In this case, the color or colors or shape or shapes of thefocus detection region 1401 and/or round part 1404 may also be changed.

The availability of distance and direction to an in-focus position maybe determined based on the degree of reliability of a focus signal. Forexample, when the degree of reliability is higher than a first thresholdvalue, it can be determined that both of the distance and direction toan in-focus position are available. When the degree of reliability isequal to or lower than the first threshold value and is higher than asecond threshold value, it can be determined that the in-focus directionis only available. When the degree of reliability is equal to or lowerthan the second threshold value, it is determined that both of thedistance and direction to the in-focus position are not available.

The focus assist display 1400 is movable to an arbitrary positiondesignated by a user by performing a touch operation on the display unit205 or using a cross key, not illustrated, for example. The focus assistdisplay 1400 may be displayed on a distinct part when a subject isdetected. For example, when a subject 1407 is detected, the focus assistdisplay 1400 is automatically placed at a position of the eye or nosebeing a distinct part of the subject 1407 (face of a human figure) asillustrated in FIG. 14C.

The focus assist display style is not limited to the style describedabove if the defocus amount and the defocus direction can be visuallyclear. An enlarged focus assist region may also be displayed.

Focus Detection Processing

Next, focus detection processing based on phase difference detection forcalculating a defocus amount in step S304 in FIG. 3 will be describedwith reference to FIG. 6. FIG. 6 is a flowchart illustrating a flow ofthe focus detection processing based on phase difference detection.

First in step S601, the focus detection signal processing unit 204obtains a pair of image signals from a focus detection region within afocus detection range set in step S302. In step S602, the focusdetection signal processing unit 204 calculates a correlation from thepair of image signals obtained in step S601. In step S603, the focusdetection signal processing unit 204 calculates an amount of change incorrelation from the correlation calculated in step S602.

In step S604, the focus detection signal processing unit 204 calculatesan amount of focus shift from the amount of change in correlationcalculated in step S603. In step S605, the focus detection signalprocessing unit 204 calculates reliability representing how much theamount of focus shift calculated in step S604 is reliable. Thereliability is a value calculated based on the matching degree andsteeping degree of the two images of the image signals, as describedabove. The processing from step S601 to step S605 is performed an equalnumber of times to the number of focus detection regions present withinthe focus detection range illustrated in FIGS. 4A to 4E.

In step S606, the camera control unit 207 converts the amount of focusshift to a defocus amount for each focus detection region. In step S607,the camera control unit 207 determines the focus detection region to beused for focus assist display, and the focus detection processing ends.

Details of Correlation Computation

Next, with reference to FIGS. 7A to 7C and FIGS. 8A to 8D, the focusdetection processing based on phase difference detection illustrated inFIG. 6 will be described in more detail.

FIGS. 7A to 7C illustrate image signals obtained from a focus detectionregion set as in FIGS. 4A to 4E. A focus detection range positions fromthe coordinates s to t, and a range necessary for a focus detectioncalculation in consideration of a shift amount positions from thecoordinates p to q. One focus detection region division positions fromthe coordinates x to y.

FIG. 7A illustrates waveforms of image signals before a shift. In FIGS.7A to 7C, an image signal A (A image) is indicated by a solid line 701,and an image signal B (B image) is indicated by a broken line 702.Regions 705 to 709 are focus detection regions as a result of thedivision in FIGS. 4A to 4E.

FIG. 7B illustrates results of a shift in the positive direction of theimage waveforms before the shift in FIG. 7A, and FIG. 7C illustratesresults of a shift in the negative direction of the image waveformbefore the shift in FIG. 7A. In order to calculate the correlation, theimage signal A 701 and the image signal B 702 are shift by 1 bit in thedirections indicated by the corresponding arrows.

Next, a method for calculating a correlation COR will be described.First, as illustrated in FIGS. 7B and 7C, the image signal A and theimage signal B are shifted by 1 bit, and a sum of absolute values of thecorresponding differences between the image signal A and image signal Bis calculated. In this case, the shift amount is indicated by i, theminimum number of shifts is indicated by p-s in FIGS. 7A to 7C, and amaximum number of shifts is indicated by q-t in FIGS. 7A to 7C. In FIGS.7A to 7C, x indicates a start coordinate of a focus detection region,and y is an end coordinate of the focus detection region. By using thesevalues, the correlation COR can be calculated by Expression (1).

$\begin{matrix}{{{C\; O\; {R\lbrack i\rbrack}} = {\sum\limits_{k = x}^{y}{{{A\left\lbrack {k + i} \right\rbrack} - {B\left\lbrack {k - i} \right\rbrack}}}}}\left\{ {\left( {p - s} \right) < i < \left( {q - t} \right)} \right\}} & (1)\end{matrix}$

FIG. 8A illustrates a waveform of the correlation. FIG. 8A is a graphplotting the shift amount along an axis of abscissae and the correlationalong an axis of ordinates. A correlation waveform 801 has regions 802and 803 around an extremum. From the graph, it can be said that thematching degree between the image A and the image B increases as thecorrelation decreases.

Next, a method for calculating an amount of change in correlation ΔCORwill be described. First, an amount of change in correlation iscalculated from a difference between correlations of every other shiftson the correlation waveform in FIG. 8A. In this case, the shift amountis indicated by i, the minimum number of shifts is indicated by p−s inFIGS. 7A to 7C, and a maximum number of shifts is indicated by q−t inFIGS. 7A to 7C. By using these values, the amount of change incorrelation ΔCOR can be calculated by Expression (2).

ΔCOR[i]=COR[i−1]−COR[i+1]{(p−s+1)<i<(q−t−1)}  (2)

FIG. 8B illustrates a waveform of the amount of change in correlationΔCOR. FIG. 8B is a graph plotting the shift amount along an axis ofabscissae and the amount of change in correlation along an axis ofordinates. The waveform 804 of the amount of change in correlation hasregions 805 and 806 where the amount of change in correlation changesfrom a positive value to a negative value. The region where the amountof change in correlation is turned to be zero is called a zero-crossing.The matching degree of the image A and the image B is the highest in thezero-crossing, and the corresponding shift amount is an amount of focusshift.

FIG. 8C illustrates an enlarged view of the region 805 in FIG. 8B whichincludes a part 807 of the waveform 804 of amount of change incorrelation. With reference to FIG. 8C, a method for calculating anamount of focus shift PRD will be described. The amount of focus shiftis first divided into an integer part β and a decimal part α. From thesimilarity of the triangle ABC and the triangle ADE in FIG. 8C, thedecimal part α can be calculated by Expression (3).

$\begin{matrix}{{{{AB}\text{:}{AD}} = {{BC}\text{:}{DE}}}{{{\Delta \; C\; O\; {R\left\lbrack {k - 1} \right\rbrack}\text{:}\Delta \; C\; O\; {R\left\lbrack {k - 1} \right\rbrack}} - {\Delta \; C\; O\; {R\lbrack k\rbrack}}} = {{\alpha \text{:}k} - \left( {k - 1} \right)}}{\alpha = \frac{\Delta \; C\; O\; {R\left\lbrack {k - 1} \right\rbrack}}{{\Delta \; C\; O\; {R\left\lbrack {k - 1} \right\rbrack}} - {\Delta \; C\; O\; {R\lbrack k\rbrack}}}}} & (3)\end{matrix}$

From FIG. 8C, the integer part β can be calculated by Expression (4).

β=k−1  (4)

As described above, from the sum of α and β, the amount of focus shiftPRD can be calculated.

In a case where a plurality of zero-crossings is present as in FIG. 8B,a zero-crossing with a high steeping degree maxder of the change incorrelation (hereinafter, called a steeping degree) is called a firstzero-crossing. The steeping degree is an index indicative of easiness offocus detection. As the value increases, the easiness of focus detectionincreases. The steeping degree can be calculated by Expression (5)below.

maxder=|ΔCOR[k−1]|+|ΔCOR[k]|  (5)

As described above, in a case where a plurality of zero-crossings ispresent, the first zero-crossing is determined based on the steepingdegree.

Next, a method for calculating the reliability of an amount of focusshift will be described. The reliability can be defined by the steepingdegree, the matching degree fnclvl of two images corresponding to theimage signals A and B (hereinafter, called two-image matching degree).The two-image matching degree is an index indicative of the precision ofan amount of focus shift. As the value decreases, the precisionincreases.

FIG. 8D illustrates an enlarged view of the region 802 in FIG. 8A whichincludes a part 808 of the correlation waveform 801. The two-imagematching degree can be calculated by Expression (6) below.

fnclvl=COR[k−1]+ΔCOR[k−1]/4

where (i)|ΔCOR[k−1]|×2≦maxder

fnclvl=COR[k]−ΔCOR[k]/4  (6)

where (ii) |ΔCOR[k−1]|×2>max der

Processing for Setting Focus Assist Display Region

Next, the processing for setting a focus assist display region in stepS302 in FIG. 3 will be described with reference to FIG. 9 and FIGS. 10Ato 10G. FIG. 9 is a flowchart illustrating a flow of the processing forsetting a focus assist display region according to this exemplaryembodiment. According to this exemplary embodiment, the size of a focusassist display region is fixed to a predetermined size smaller than aminimum detectable face size though the size of the focus assist displayregion may be changeable. The predetermined size may be a minimum sizein which focus detection can be performed.

First, in step S901, the camera control unit 207 determines whether thesubject detecting unit 210 has detected a face of a human figure or notin the processing in step S301. If a face of a human figure has beendetected, the processing advances to step S902. If not, the processingadvances to step S903.

In step S902, the camera control unit 207 determines whether the size ofthe detected face is larger than a predetermined threshold value TH1 ornot. If it is determined that the size of the detected face is largerthan the threshold value TH1, the processing advances to step S904. Ifit is determined that the size of the detected face is equal to or lowerthan the threshold value TH1, the processing advances to step S911.

Here, the predetermined threshold value TH1 is defined such that a focusassist frame set in accordance with the position of one of the right andleft eyes (distinct region) cannot fit within a face frame if the sizeof the face is equal to or lower than the threshold value TH1. Themethod for determining the threshold value TH1 may be assumed as a valueapproximately two times the size in the horizontal direction of thefocus assist frame, for example through it is not limited thereto. Ingeneral, in a case where a human figure is imaged, focus adjustment ofan eye of a human figure (one of the right and left eyes because thefocus position differs between the right and left eyes). Therefore, whena human figure is imaged in the MF mode, the focus assist display may beimplemented in a region having one of the right and left eyes. Then, auser can save the time and labor for performing an operation for movingthe focus assist frame to the eye region.

However, in a case where a face is present at a far position from acamera or where the face of a human figure within an image is smallbecause the focal length is in the wide-angle direction, a difference infocus between the eye and another part is difficult to be recognized.FIG. 10A illustrates an example in a case where the size of the detectedface (corresponding to a face frame 1001) is small. When a focus assistframe 1002 having a size settable under control is set in accordancewith the position of one eye (right eye in this case) as illustrated inFIG. 10A, the proportion of the background occupying the frame may belarge. This may cause an erroneous detection, and the possibility mayincrease in which the focus assist display may vary between times or auser may improperly determine a focusing state as an in-focus state.

If the size of the focus assist frame settable under control can be setin accordance with an eye region, the detection of a defocus amount isdifficult because the focus detection range is significantly small.Therefore, it is not suitable for a case where focus adjustment isperformed by performing an MF operation by checking changes in the stateof the assist display from a largely blurred state.

Accordingly, when the size of the detected face is equal to or lowerthan a predetermined threshold value, a focus assist frame 1003 is setso as to fit into the face frame 1001, as illustrated in FIG. 10B, instep S911. In this case, the focus assist frame 1003 may be set so as toinclude the eye region for higher contrast of the image signals. Settingthe focus assist frame 1003 in FIG. 10B within the face frame 1001 canprevent an influence of the background. Thus, the possibility ofoccurrence of an erroneous detection can be reduced, and the in-focusstate can be recognized accurately by a user. When the size of the focusassist frame is changeable, a focus assist frame may be set in a regionsubstantially identical to the face frame or a region having a sizeequal to or higher than a predetermined ratio within the face frame.

In order to detect the position of the eye from the face of a humanfigure, a (relative) position at a general ratio from the size andposition of the detected face (such as a position at ⅓ of a length ofthe face from an upper position of the face) may be defined as theposition of the eye. Alternatively, the subject detecting unit 210 mayperform a publicly known detection process on an image pickup signaloutput from the CDS/AGC converter 202 to detect a specific region (ofthe eye, ear, nose, or mouth, for example) within an image region of adetected face.

On the other hand, if the size of the detected face is larger than thepredetermined threshold value TH1, the camera control unit 207 in stepS904 receives direction information of the detected face from thesubject detecting unit 210 and determines the direction of the face. Ifit is determined that the direction of the face is forward, theprocessing moves to step S905. If it is determined that the direction ofthe face is sideways, the processing moves to step S906. It may bedetermined that the direction of the face is forward if the angle of theface about the camera (imaging optical system) is equal to or within apredetermined angle.

In step S905, the camera control unit 207 receives position informationof the detected face from the subject detecting unit 210 and determineswhich of the right and left sides the position of the face is on aboutthe center of the horizontal direction of an imaging screen. If theposition of the face is on the left side of the screen, the processingmoves to step S907. If the position of the face is on the right side ofthe screen, the processing moves to step S908.

In step S907, the camera control unit 207 sets a focus assist frame atthe position corresponding to the right eye of the human figure. As anexample of a specific setting method, the centroid position of a focusassist frame may be set at the centroid position of the right eyeregion, though it may be set in accordance with other references. Afterthe focus assist frame is set at the position corresponding to the righteye of the human figure, the processing moves to step S914.

On the other hand, the camera control unit 207 sets the focus assistframe at a position corresponding to the left eye of the human figure instep S908, and the processing then moves to step S914. Also in thiscase, the centroid position of the focus assist frame may be set at thecentroid position of the left eye region or may be set in accordancewith other references.

The reason will be described why the focus assist frame is set at aposition corresponding to the right eye or left eye of the human figurein accordance with the position of the face within a screen in step S907and step S908 if the direction of the human figure is forward.

FIG. 10C illustrates an example of a scene where a human figure faces tothe front. When the direction of a human figure is forward, the humanfigure also faces toward the camera. Therefore, there is a highpossibility that the eye farther away from the center of the screen ispresent on the minimum-object-distance side on a focal plane. Generally,the focus may often be adjusted to the eye on theminimum-object-distance side of a human figure, the focus assist frameis set at a position corresponding to the eye farther away from thecenter of the screen according to this exemplary embodiment. This cansave the time and labor of a user for performing an operation for movingthe focus assist frame to the position of the eye on theminimum-object-distance side.

For the reason above, when a human figure is on the right-hand side(position 1004) of a camera, the focus assist frame 1006 is set on theleft eye of the human figure. When the human figure is on the left side(position 1005) of the camera, the focus assist frame 1007 is set on theright eye of the human figure.

On the other hand, in step S906, the camera control unit 207 receivesinformation regarding the direction of the detected face from thesubject detecting unit 210 and determines the direction of the face.Here, if the right side of the face directs toward the screen, it isdetermined that the direction of the face is rightward. If the left sideof the face directs toward the screen, it is determined that thedirection of the face is leftward. If the direction of the face isrightward, the processing moves to step S909. If the direction of theface is leftward, the processing moves to step S910.

In step S909, the camera control unit 207 sets the focus assist frame ata position corresponding to the right eye of the human figure, like stepS907. The processing then moves to step S914. In step S910, the cameracontrol unit 207 sets the focus assist frame at a position on the lefteye of the human figure, like step S908. The processing moves to stepS914.

The reason will be described why the focus assist frame is set on theright eye or left eye of a human figure in accordance with the directionof the face of a human figure in step S909 and step S910.

FIGS. 10D and 10E illustrate a state in which the direction of a humanfigure is leftward. First of all, as illustrated in FIG. 10D, setting afocus assist frame 1008 at a position corresponding to the right eyewhen the direction of the human figure is leftward increase theproportion of the background occupying the region, like a case where thesize of the face is small. Thus, erroneous detections may easily occur,and there is an increased possibility that the focus assist display mayvary from time to time. Because of this, there is an increasedpossibility that a user may not easily adjust the focus lens to anin-focus position by performing an MF operation or may determine that anin-focus state is obtained at an improper focus lens position.

When the direction of a human figure is leftward, it means the left eyeis on the minimum-object-distance side. Accordingly, setting the focusassist frame 1009 at the position corresponding to the left eye on theminimum-object-distance side as illustrated in FIG. 10E can save user'slabor and time for performing an operation for moving the focus assistframe to the position corresponding to the eye on theminimum-object-distance side.

In step S914, the camera control unit 207 stores information regardingthe position of the focus assist frame set in step S907 to S911 about aregion (face frame) of the face. Information regarding the proportion ofthe size of the focus assist frame about the region (face frame) of theface may also be stored.

On the other hand, when no face has been detected in step S901, theprocessing moves to step S903 where the camera control unit 207determines whether a face tracking mode is enabled or not. In the facetracking mode, when a state where a face is being detected shifts to astate where no face is detected, a region highly possibly correspondingto a face is being estimated from information immediately before thestate where no face is detected and an amount of characteristics of ageneral face. If the camera control unit 207 determines that the facetracking mode is enabled, the processing moves to step S912. If it isdetermined that the face tracking mode is not enabled, the processingmoves to step S913.

In step S912, the camera control unit 207 reads out informationregarding the position of the focus assist frame stored in step S914about the region corresponding to the face. The stored position isconverted to the position about the region (tracking region)corresponding to the face being tracked, and the focus assist frame isset based on the position. When the size of the focus assist frame ischangeable, information regarding the proportion of the size of thefocus assist frame to the region corresponding to a face may be storedin step S914 and be read out to set the size of the focus assist frame.In this case, the stored size (the proportion of the size to the faceframe) to the size to the tracking region to set the focus assist frame.

The reason why the processing in step S912 is performed will bedescribed with reference to FIGS. 10F and 10G. FIGS. 10F and 10Gillustrate a scene in which a human figure whose face is sideways andthe face is no longer detected, resulting in the face tracking mode. Asillustrated in FIG. 10F, when a focus assist frame 1011 is set at thecenter of a tracking region 1010, there is a high possibility that thefocus assist frame 1011 comes off the position of the eye of the humanfigure. Accordingly, as illustrated in FIG. 10G, a focus assist frame1012 may be set based on the position of the focus assist frameimmediately before the face tracking mode is enabled so that thepossibility can increase in which the focus assist frame 1012 containsthe eye of a human figure. When the human figure at the state in FIG.10G turns to the camera again, the face may be detected again. Thus, thefocus assist frame can be smoothly kept set at the position of the eye.

On the other hand, in step S913, the camera control unit 207 sets thefocus assist frame at a prestored position. Then, the processing ends.According to this exemplary embodiment, the prestored position is thecenter of the screen. However, an embodiment of the present invention isnot limited thereto. If a face is detected for a predetermined period oftime in neighborhood of the position subject to the tracking or if thereis a low possibility that the detected subject is a face because of alower amount of facial characteristics in the face tracking mode, theface tracking mode ends. When the face tracking mode ends and if a faceis detected at a position different from the position subject to theface tracking, a focus assist frame may be set based on the detectedface. If no other face is detected when the face tracking mode ends, afocus assist frame may be set a predetermined position and sizeprestored in the camera control unit 207, the position and size at thetime when the face tracking mode ends may be held for setting.

According to this exemplary embodiment, when face detection is beingperformed in step S901 or when the face tracking mode is enabled in stepS903, that is, when facial information is being obtained, a focus assistframe is set within a face frame (which will be called “facial MF mode”,for example). On the other hand, when face detection is not beingperformed and the face tracking mode is not enabled, that is, whenfacial information is not being obtained, a focus assist frame is setbased on a prestored position (“normal MF mode”, for example).

Processing for Changing Focus Assist Display Region

Next, processing for changing a focus assist display region in step S303in FIG. 3 will be described with reference to FIG. 13. FIG. 13 is aflowchart illustrating a flow of the processing for changing a focusassist display region.

First, in step S1301, the camera control unit 207 receives a changeoperation through the camera operating unit 208 and determines whether achange operation has been performed or not. The change operation here isassumed as an input operation with a cross key or a touch operation ontothe display unit 205, for example, the operating member and the form ofoperation are not limited thereto. If a change operation is performed,the processing moves to step S1302. If a change operation is notperformed, the flow ends.

In step S1302, the camera control unit 207 determines whether the facialMF mode is enabled or not. If the facial MF mode is enabled, theprocessing moves to step S1303. If not, the processing moves to stepS1305.

In step S1303, the camera control unit 207 determines whether thesubject detecting unit 210 has detected a plurality of faces in theprocessing in step S301 or not. If so, the processing moves to stepS1304. If not, the flow ends.

In step S1304, the camera control unit 207 performs processing forchanging the main face for which a focus assist frame is to be set inaccordance with the operation detected in step S1301. In other words,the focus assist frame is moved to a face excluding the face for whichthe current focus assist frame is displayed. After performing theprocessing for changing the main face, the flow ends.

On the other hand, in step S1305, the camera control unit 207 performsprocessing for moving the position of the focus assist frame inaccordance with the operation detected in step S1301. For example, thefocus assist frame may be moved by a predetermined amount in thedirection operated on the cross key, or the focus assist frame is movedto the position designated by a touch operation on the display unit 205.After performing the processing for moving the position of the focusassist frame, the flow ends.

As described above, according to this exemplary embodiment, if a changeoperation is performed through the camera operating unit 208 while focusassist display is being implemented, the processing is changed inaccordance with the state related to obtaining of facial-information.More specifically, if information regarding a plurality of faces isobtained, processing for changing the face for which the focus assistframe is to be set is performed. If facial information is not obtained,the processing for moving the position of the focus assist frame isperformed in accordance with the change operation. This is because thefocusing state of one eye may be displayed when facial information isobtained also if a user operates the camera operating unit 208 in orderto change the focus assist frame position, as described above.

Having described the setting of a focus assist frame in the MF mode, thefocus assist frame is not set in the AF mode but an AF frame forobtaining a signal for performing focus adjustment is set. Here, when anAF frame is set on a region for one eye if facial detection is performedin the AF mode, the size of the AF frame is reduced. Thus, when asubject moves, the association of the subject with the AF frame mayeasily change, and the AF is highly possibly instable. Accordingly, inthe AF mode, stable AF can be achieved by setting the AF frame for aface region (for example, the AF frame may be set to have a sizesubstantially equal to that of the face frame). For the reason above,even when the operation for changing the size of the AF frame isperformed through the camera operating unit 208, the minimum size of theAF frame is set to a size larger than the focus assist frame.

Also when the size of the focus assist frame is fixed, the size of theAF frame is changeable in the AF mode. This is because a smaller focusdetection range may be set for performing precise focus adjustment by amanual operation in the focus assist display while the AF frame may beset in accordance with the size of a subject in the AF.

As described above, according to this exemplary embodiment, the settingof the focus assist frame is changed in accordance with the staterelated to obtaining of subject information. For example, in a casewhere information related to a predetermined subject (such as the face)is obtained, the focus assist frame is set within a region correspondingto the predetermined subject in consideration of the focus adjustment tobe performed by an MF operation on the predetermined subject.Particularly when information related to a subject (such as the face)having a plurality of feature parts (such as the eyes) is obtained, afeature part on the minimum-object-distance side is determined based onthe state of the subject for implementing focus assist display. Thus, anindicator (focus assist display) representing a focusing state can bedisplayed at a proper position about the subject so that user'sconvenience can be improved.

According to this exemplary embodiment, when a user performs anoperation for changing the focus assist frame, processing is performedbased on a state related to obtaining of subject information. Forexample, when a plurality of pieces of information regarding apredetermined subject is detected, processing is performed for changingthe subject for which the focus assist frame is set in accordance with auser's operation. Thus, the user's intention for the changing can bereflected in a proper manner for the subject to set the focus assistframe.

Problems of Stability and Responsiveness of Focus Assist Display

A problem that stable focus detection results are not obtained due tochanges of the picture within the focus detection region 1401 in a casewhere a user moves the focus detection region 1401 or where the focusassist display 1400 is implemented automatically within a subject, forexample. In this case, the angle display part 1402 and the directiondisplay part 1405 do not change smoothly, which makes a user feel asense of unease regarding his or her focus operation. Accordingly,results of a plurality of focus detections may be averaged for use toimprove the stability of the display 1400. However, though the displayof the angle display part 1402 and the direction display part 1405becomes stable, the use of such an average value may impair theresponsiveness because data regarding the subject which have beendetected in the past are used in a case where the subject is changedwhen, for example, the focus detection region is moved.

Focus Assist Display Control Based on Moving Amount of Focus AssistFrame

A method for changing the focus assist display control based on anamount of change (moving amount) of a position of a focus detectionregion will be described below with reference to FIG. 15. For example,according to exemplary embodiment, contents (items) of focus assistdisplay, the average number of focus detection results to be used forangle display, and the position of the focus detection region arechanged. The focus assist display control is changed in a case wherefocus assist display is moved manually and in a case where focus assistdisplay is moved automatically (while a subject is being detected).Manual movement of a focus detection region roughly includes threepatterns. The amount of change of the position of the focus detectionregion is determined based on an overlap rate of the focus detectionregion between the last detection of the focusing state and thisdetection of the focusing state. When the overlap rate is 100% to 80%(the amount of change is equal to or lower than a second amount), “nomovement” is determined. When the overlap rate is 80% to 50% (the amountof change is higher than the second amount and equal to or lower than afirst amount), “small movement (low speed)” is determined. When theoverlap rate is 50% to 0% (the amount of change is higher than the firstamount), “large movement (high speed)” is determined. Because focusdetection is performed repeatedly in a predetermined period, a casewhere the amount of change (moving amount) of the focus detection regionis large can be said as a case where the speed of movement of the focusdetection region is high. As described above, according to thisexemplary embodiment, the change in position of the focus detectionregion between the last focus detection and the present focus detectionis determined, and the focus assist control is changed based on thedetermination result, which will be described below. More specifically,the responsiveness of the focus assist display is increased as thechange in position of the focus detection region increases.

With “no movement” and “small movement”, results of a plurality ofdetections of the focusing state are averaged, and the focus assistdisplay showing the “angle” and the “direction” is implemented based onthe average result. The “angle” here corresponds to the defocus amount,and the “direction” corresponds to the direction toward an in focusstate. For “small movement”, the number of times of averaging ofdetection results of the focusing state may be reduced, compared withthat for “no movement”, to increase the responsiveness of the focusassist display.

As illustrated in FIG. 15, for “no movement”, the contents of thedisplay are “angle” and “direction”, and the number of times ofaveraging of focus detection results to be used for angle display is setas 10 for implementing stable display. For “small movement”, thecontents of the display are “angle” and “direction”, and the number oftimes of averaging of focus detection results to be used for angledisplay is set as 5 for emphasizing the responsiveness more than stabledisplay, compared with “no movement”.

For “large movement”, the content of the display is “direction” only,and no averaging of focus detection results to be used for angle displayis performed. This is because averaging focus detection results for aplurality of periods may possibly result in use of focus detectionresults corresponding to a different subject in the past when the movingamount is large. Therefore, instead of the averaging of focus detectionresults, a value acquired based on a focus signal obtained from thefocus detection region after the movement is directly used for focusassist display. However, in this case, the direction is only displayedwithout display of the angle because the angle display indicative of thedistance (defocus amount) to an in-focus position may be instable. Thisis because, for “large movement”, there is a possibility that a user haschanged the subject and the complexity on the screen therefore can bereduced without displaying the angle for preventing instable andunnecessary display.

On the other hand, when the focus detection region is movedautomatically upon detection of a subject, for example, the contents ofthe display are “angle” and “direction”. Focus detection resultscorresponding to a plurality of focus detection regions set as will bedescribed below within the detected subject are used for displaying theangle and the direction, without performing the processing forcalculating the average of focus detection results to be used for angledisplay.

As described above, when whether the distance and direction to thein-focus position are available or not is determined based on the degreeof reliability of the focus signal, the determination result is used bypriority to determine the content or contents of the display. In otherwords, when both of the distance and direction to an in-focus positionare not available, it is indicated that the focus detection is invalid.When the direction to an in-focus position is only available, thecontent of the display is “direction” only. When both of the distanceand direction to an in-focus position are available, the content orcontents of the display is or are determined in accordance with theamount of change of the position of the focus detection region, asillustrated in FIG. 15. For “no movement” and “small movement”, if thenumber of times of determination that the distance to an in-focusposition of the number of times of averaging focus detection results isequal to or higher than a predetermined number of times, “angle” (and“direction”) may be displayed. In this case, the number of times ofdetermination that the distance to an in-focus position is available islower than the predetermined number of times, “direction” may only bedisplayed.

This is because, if a subject such as a face is detected, substantiallyno distance change may occur within a face, the precision of the focusdetection can be increased by using the focus detection results from aplurality of focus detection regions within the corresponding faceframe.

When a subject such as a human figure is detected, it may be expectedthat the subject may move around within the screen. Thus, application ofthe same display method as that for a case where a focus detectionregion is manually moved may result in easy occurrence of a case whereTHE moving amount of the focus detection region is determined as “largemovement” and the angle display is disabled. Therefore, the focus assistdisplay while a subject is detecting includes angle display anddirection display using focus detection results from a plurality offocus detection regions defined within a subject detection frame. Thus,stable angle display using a plurality of focus detection results andimproved responsiveness without performing the averaging process canboth be implemented.

When a human figure is a subject, the focus detection region to be usedmay be changed in accordance with the angle of the face of the subjectso that the precision of the focus assist display can further beimproved.

FIGS. 16A and 16B are flowcharts illustrating a procedure of focusassist display control to be executed by the camera control unit 207.This processing is executed in a predetermined period in accordance witha computer program stored within the camera control unit 207. Forexample, the processing may be executed in read cycles (every verticalsynchronization period) of an image pickup signal from the image sensingelement 201 for generation of an image for one frame (or one field) ormay be repeated a plurality of number of times within a verticalsynchronization period.

First of all, in step S1601, the camera control unit 207 determineswhether a manual focus operation is enabled or not. If the manual focusoperation is enabled, the processing moves to step S1602. If not, theprocessing ends without performing anything. In step S1602, the cameracontrol unit 207 performing processing for setting a focus detectionregion.

The processing for setting a focus detection region to be performed instep S1602 is performed in the same manner as in FIGS. 4A to 4E. Withreference to FIG. 17, a flow of the processing for setting a focusdetection region will be described.

FIG. 17 is a flowchart illustrating an example of the processing forsetting a focus detection region. First of all, in step S1701, thecamera control unit 207 determines whether the position of the focusdetection region has been designated or not is determined. If theposition of the focus detection region has been designated, theprocessing moves to step S1705. If not, the processing moves to stepS1702. The position may be designated by a user by operating a touchpanel or a cross key, not illustrated, for example, though how theposition is designated is not described in detail here because it isdifferent from the gist of the present invention. In step S1705, thecamera control unit 207 sets a focus detection region at the designatedposition as illustrated in FIG. 4E.

In step S1702, the camera control unit 207 determines whether a functionfor detecting a subject (face detection function here) is ON or OFF. Ifthe function for detection of a subject is ON, the processing moves tostep S1704. If not, the processing moves to step S1703. In step S1703,because the face detection is not performed and the position of thefocus detection region is not designated, the camera control unit 207sets the focus detection region at a predetermined position (such as thecenter). In step S1704, the camera control unit 207 sets the focusdetection region at the position of the detected face as illustrated inFIG. 4D. When no face is detected, the focus detection region may be setat a predetermined position as in step S1703.

In step S1704, the processing (processing for setting a focus assistframe in accordance with the size, direction, and position of the face)as a result of determination of Yes in S901 in FIG. 9 may be performed.In the face tracking mode, the processing in S912 may be performed.

How the focus detection region is to be arranged and the size of theregion, for example, are not limited to the examples illustrated inFIGS. 4A to 4E and FIG. 17, but may be set without departing from thespirit and scope of the invention.

After the setting of the focus detection region in step S1602 ends, theprocessing moves to step S1603. In step S1603, the camera control unit207 determines whether the focus detection region is set at the positionof the face detected by the subject detecting unit 210. If so, theprocessing moves to step S1612. If not, the processing moves to stepS1604.

In step S1604, focus detection processing is performed. The focusdetection processing to be performed here is performed in the samemanner as in the FIG. 6 to FIGS. 8A to 8D.

After the focus detection processing ends in step S1604, the processingmoves to step S1605. In step S1605, the camera control unit 207determines the moving amount of the focus detection region. The movingamount is determined based on the overlap rate of the focus detectionregion between the last detection of the focusing state and the currentdetection of the focusing state, as described with reference to FIG. 15.When the overlap rate is 100% to 80%, “no movement” is determined. Whenthe overlap rate is 80% to 50%, “small movement” is determined. When theoverlap rate is 50% to 0%, “large movement” is determined. If “largemovement” is determined, the processing moves to step S1606. When “smallmovement” is determined, the processing moves to step S1607. If “nomovement” is determined, the processing moves to step S1608. The methodfor determining the moving amount is not limited to the method describedabove, but the moving amount may be determined based on a change ofcoordinates of the center position of the focus detection region betweenthe last focus detection and the current focus detection, for example. Acase where there is a movement of the focus detection region correspondsto a case where there is a movement of the focus assist frame in stepS1305 in FIG. 13.

In step S1606, step S1607, and step S1068, the camera control unit 207acquires the average of the focus detection results. The acquiredaverage number of times is used as a parameter for processing. Withreference to FIG. 18, the processing for averaging focus detectionresults to be performed in step S1606, step S1607, and step S1608 inFIG. 16B will be described below.

In step S1801, the camera control unit 207 obtains the average number oftimes Th_A illustrated in FIG. 15, The processing then moves to stepS1802. In step S1802, the camera control unit 207 determines whether theaverage number of times Th_A is equal to 0 or not. If the average numberof times Th_A is equal to 0, the processing moves to step S1803. In stepS1803, the current focus detection result (with respect to the defocusamount and the degree of reliability) is set as data to be used forfocus assist display, without performing the processing of averaging.

On the other hand, if the average number of times Th_A is not equal to0, the processing moves to step S1804 where the camera control unit 207divides the sum total of the defocus amounts obtained by the latest Th_Afocus detections by Th_A to calculate the average of the defocusamounts. Next, in step S1805, the camera control unit 207 divides thesum total of degrees of reliability obtained by the latest Th_A focusdetections by Th_A to calculate the average of the degree ofreliability.

In step S1806, the camera control unit 207 sets the averages of thefocus detection results (defocus amount and degree of reliability)obtained in step S1804 and step S1805 as data to be used for focusassist display. After the processing for averaging the focus detectionresults ends as described above, the processing returns to theprocessing in FIGS. 16A and 16B.

After the processing in step S1607 and step S1608, the camera controlunit 207 in step S1609 performs processing for calculating an angleindex indicative of the display position of an angle display part to beused for focus assist display. If the moving amount is determined as“large movement” in step S1605, the processing moves to step S1610without calculating the angle index after the processing in step S1606because the angle display part is not to be displayed, as described withreference to FIG. 15.

With reference to FIG. 19, the processing for calculating an angle indexto be performed in step S1609 will be described below. In step S1901,the camera control unit 207 determines whether the average of degrees ofreliability calculated in step S1607 or step S1608 is equal to or higherthan a predetermined threshold value TH_B. This determination isperformed for determination of whether an angle index can be calculatedby using the average of defocus amounts calculated in step S1607 or stepS1608. If the degree of reliability is equal to or higher than thethreshold value TH_B, the defocus amount is determined as a reliablevalue. Then, the processing moves to step S1902. If it is lower than thethreshold value TH_B, it is determined that there is a possibility thatthe defocus amount is not reliable, and the processing moves to stepS1904.

In step S1902, the camera control unit 207 converts the defocus amountto an angle. According to an example of the conversion, how many timesthe defocus amount is larger than the focal point depth where the focalpoint depth is 1° to calculate the angle index. The method forconversion to an angle is not limited to the calculation method but mayinclude changing in accordance with the sensitivity of a focus ring, forexample.

In step S1903, the camera control unit 207 sets a flag indicating thatthe angle display part is to be displayed. In step S1904, the cameracontrol unit 207 sets a flag indicating that the angle display part isnot to be displayed. The processing then moves to step S1610 in FIG. 16.

In step S1610, the camera control unit 207 performs processing forcalculating a direction index indicative of the display direction of thedirection display part to be used for focus assist display. Withreference to FIG. 20, the processing for calculating a direction indexto be performed in step S1610 will be described below. In step S2001,the camera control unit 207 determines whether the degree of reliabilityacquired in step S1606, step S1607, or step S1608 is equal to or higherthan a predetermined threshold value TH_C or not. This determination isperformed for determining whether a direction index can be calculated byusing the defocus amount calculated in step S1606, step S1607, or stepS1608. If the degree of reliability is equal to or higher than thethreshold value TH_C, the direction calculated from the defocus amountis determined as a reliable value. The processing then moves to stepS2002. If it is lower than the threshold value TH_C, it is determinedthat there is a possibility that the direction calculated from thedefocus amount may not be reliable. The processing then moves to stepS2004. The threshold value TH_B is a threshold value indicating a higherdegree of reliability than the threshold value TH_C.

In step S2002, the camera control unit 207 converts the defocus amountto a direction. As an example, a direction is calculated from the signof the defocus amount. The camera control unit 207 sets a flagindicating that direction display is to be performed in step S2003 andsets a flag indicating that direction display is not to be performed instep S2004. The processing then moves to step S1611 in FIG. 16.

In step S1611, the camera control unit 207 performs focus assist displayprocessing. This processing may display a part necessary for focusassist display and display an indicator of a focusing state and anindicator of disability of focus detection described with reference toFIGS. 14A to 14C on the display unit 205 based on the flags for theangle display and the direction display and the calculated angle indexand direction index.

On the other hand, in a case where a focus detection region is set at aposition of the face detected by the subject detecting unit 210 in stepS1603, the processing moves to step S1612 where the camera control unit207 determines the direction of the face. If it is determined that thedirection of the face is sideways, the processing moves to step S1613.If it is determined that direction of the face is forward or backward,the processing moves to step S1614.

In step S1613 and step S1614, the camera control unit 207 additionallyset the focus detection regions. Here, as illustrated in FIGS. 21A and21B, a focus detection region to be used is additionally set inaccordance with the direction of the face. FIG. 21A illustrates a methodfor setting focus detection region if the direction of the face isforward. Focus detection regions 2102, 2103, and 2104 are set on a faceforward 2101. For a face forward, if the focus detection region 2102 isset in step S1602, for example, the focus detection regions 2103 and2104 are additionally set in horizontally. Additionally setting a focusdetection region within a region corresponding to the same face as thatfor the focus detection region 2102 set in step S1602 can definesubstantially equal distances between the focus detection region 2102and the additionally set focus detection regions 2103 and 2104. Asdescribed above, a plurality of focus detection regions are laid outhorizontally, and focus assist display is thus implemented by using aplurality of focus detection results therefrom.

On the other hand, FIG. 21B illustrates a method for setting focusdetection regions if the face is sideways. Focus detection regions 2106,2107, and 2108 are set on a face sideways 2105. When the face issideways and if the focus detection region 2107 is set in step S1602,for example, the focus detection region 2106 and 2108 are additionallyset vertically. Additionally setting a focus detection region within aregion corresponding to the same face as that for the focus detectionregion 2107 set in step S1602 can define substantially equal distancesbetween the focus detection region 2107 and the additionally set focusdetection regions 2106 and 2108. For a face sideways, there is a highpossibility that horizontally placed focus detection regions may be offthe region of the face, focus detection regions are laid out vertically.As described above, a plurality of focus detection regions are laid outvertically, and focus assist display is implemented by using a pluralityof focus detection results therefrom. According to this exemplaryembodiment, three focus detection regions are provided. However, it isgiven for illustration purpose only, and the number of focus detectionregions is not limited thereto.

In step S1615, focus detection processing is performed as describedabove with reference to FIG. 6 to FIG. 8D for each of plurality of focusdetection regions. In step S1616, calculation processing is performedbased on the plurality of focus detection results acquired in stepS1615. More specifically, the camera control unit 207 averages theplurality of focus detection results calculated in step S1615. Averagingis applied according to this exemplary embodiment, a focus detectionresult to be used may be selected in consideration of the degree ofreliability of the focus detection result, for example.

Also in the display control illustrated in FIGS. 16A and 16B, if a focusdetection region corresponds to a face (Yes in step S1603), the changeoperation described with reference to FIG. 13 may be received. In otherwords, if the change operation is performed through the camera operatingunit 208, the processing moves to step S1303 in FIG. 13 where, if aplurality of faces are detected, the processing for changing the mainface may be performed.

After processing the focus detection results, the processing describedabove is performed in step S1609 and step S1610. The camera control unit207 in step S1611 then performs the focus assist display processing.

Under the display control described with reference to FIGS. 16A and 16B,the stability of the focusing state display can be kept and at the sametime the responsiveness can be improved in the focus assist displaywhile manual focus adjustment is being performed. Having described that,according to this exemplary embodiment, the focus assist display isimplemented on the display unit 205 included in the camera main body 20,the focus assist display may be implemented by using informationobtained from the camera main body 20 on a display unit separate fromthe camera main body 20.

Second Exemplary Embodiment

Next, a second exemplary embodiment will be described with reference toFIG. 11 and FIGS. 12A and 12B. This exemplary embodiment is differentfrom the first exemplary embodiment in that a setting for focus assistdisplay is changed based on the depth of field. Like numbers refer tolike parts in the first and second exemplary embodiments, and therepetitive description will be omitted.

FIG. 11 is a flowchart illustrating a flow of processing for setting afocus assist display region according to this exemplary embodiment,instead of FIG. 9. According to this exemplary embodiment, the size of afocus assist display region is changeable.

First of all, in step S902, if the size of the detected face is largerthan a threshold value TH1, the processing moves to step S1101. In stepS1101, the camera control unit 207 determines whether the depth of fieldis smaller than a threshold value TH2 or not. If the depth of field issmaller than the threshold value TH2, the processing moves to step S904.Then, the same processing as that in step S901 is performedsubsequently.

On the other hand, if the depth of field is equal to or larger than thethreshold value TH2, the processing moves to step S1102 where a focusassist frame is set based on the entire face frame. For example, a focusassist frame is set in a region substantially equal to the correspondingface frame or in a region having a size equal to or larger than apredetermined ratio within the face frame. After the focus assist frameis set in step S1102, the processing moves to step S914 where the sameprocessing as in FIG. 9 is performed. If it is determined that the sizeof the face detected in step S902 is equal to or larger than thethreshold value TH1, the processing in step S1102 may be performed.

According to this exemplary embodiment as described above, even if thesize of the detected face is larger than the threshold value TH1 and ifthe depth of field is equal to or larger than a threshold value, thefocus assist display is implemented based on the entire face frame. Thereason will be described why the setting of a focus assist frame ischanged based on the depth of field.

If the depth of field is shorter than the length in depth of thedetected face (such as the tip of the nose to the back of the head),there is a high possibility that the difference in focusing betweenparts of the face may be identified. For example, when the direction ofthe face is rightward, a difference in distance to the camera betweenthe left eye and the right eye is equal to the depth of the face. Thus,if the depth of field is shorter than the length of the depth of theface, the difference in focusing between the left eye and the right eyemay be visually recognizable. Therefore, a focus assist frame is set onthe right eye. FIG. 12A illustrates a state in which a focus assistframe 1202 is set in a region corresponding to the right eye within aface frame 1201.

On the other hand, if the depth of field is longer than the length ofthe depth of the face, there is a high possibility that the differencein focusing between parts of the face may not be identified. Forexample, when the direction of the face is rightward and is sideways andif the depth of field is longer than the length of the depth of theface, the difference in focusing between the left eye and the right eyemay not be visually recognized. Accordingly, by giving priority to thedetection of a defocus amount with higher precision, a focus assistframe 1203 may be set based on the entire face frame 1201, asillustrated in FIG. 12B.

According to this exemplary embodiment as described above, the settingof the focus assist frame is changed based on the depth of field. Morespecifically, when the depth of field is shorter, a focusing state isdetected for display from a smaller region for precise focus adjustment.On the other hand, when the depth of field is longer, a focusing stateis detected for display from a larger region by giving priority to theprecision of focus detection. By performing the processing, focus assistdisplay with higher convenience can be implemented.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-077205, filed Apr. 3, 2015, and Japanese Patent Application No.2015-077146, filed Apr. 3, 2015, which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. A focus detection apparatus comprising: an image pickup unit configured to perform photoelectric conversion on a luminous flux having passed through an imaging optical system; a focus detection unit configured to detect a focusing state based on a signal generated by the image pickup unit; a setting unit configured to set a first region within an image generated by the image pickup unit; a display controller configured to control such that an index representing the focusing state detected by the focus detection unit within the first region can be superimposed on the image; and an obtaining unit configured to obtain information regarding a predetermined subject in the image, wherein the setting unit sets a position of the first region based on information regarding at least one of a position, a size, and a direction of the predetermined subject in the image.
 2. The focus detection apparatus according to claim 1, wherein the predetermined subject is a subject having a plurality of distinct regions; and wherein the setting unit sets a position of the first region based on a position of a distinct region of the plurality of distinct regions.
 3. The focus detection apparatus, according to claim 2, wherein, in a case where the size of the predetermined subject is larger than a first threshold value, the setting unit sets a position of the first region based on a position of a distinct region of the plurality of distinct regions included in the predetermined subject, and, in a case where the size of the predetermined subject is equal to or smaller than the first threshold value, the setting unit sets a position of the first region based on a position of the predetermined subject.
 4. The focus detection apparatus according to claim 2, wherein the setting unit sets a position of the first region based on a position of a distinct region far away from the center of the image among the plurality of distinct regions.
 5. The focus detection apparatus according to claim 2, wherein the setting unit sets a position of the first region based on a position of the distinct region on a most minimum-object-distance side among the plurality of distinct regions.
 6. The focus detection apparatus according to claim 5, wherein the setting unit determines the distinct region on a most minimum-object-distance side among the plurality of distinct region based on the direction of the predetermined subject.
 7. The focus detection apparatus according to claim 1, further comprising: an operating member configured to receive a predetermined operation, wherein the setting unit changes the position of the first region in accordance with the predetermined operation.
 8. The focus detection apparatus according to claim 7, wherein, if information regarding a plurality of predetermined subjects is obtained, the setting unit sets the first region based on information of one of the predetermined subjects and changes the predetermined subject for which the first region is set in accordance with the predetermined operation.
 9. The focus detection apparatus according to claim 7, wherein the predetermined operation is an operation for designating a direction; and wherein the setting unit moves the position of the first region by a predetermined amount in a direction based on the predetermined operation.
 10. The focus detection apparatus according to claim 7, wherein the predetermined operation is an operation for designating a position; and wherein the setting unit changes the position of the first region to the position based on the predetermined operation.
 11. The focus detection apparatus according to claim 7, wherein the display controller increases responsiveness of display of the index in a case where the amount of change in position of the first region in accordance with the predetermined operation is equal to a first amount, compared with a case where the amount of change is equal to a second amount smaller than the first amount.
 12. The focus detection apparatus according to claim 7, wherein the display controller displays the index based on a latest detection result of the focusing state by the focus detection unit in a case where the amount of change in position of the first region in accordance with the predetermined operation is equal to a first amount, and display the index based on an average of a plurality of detection results of the focusing state by the focus detection unit in a case where the amount of change is equal to a second amount smaller than the first amount.
 13. The focus detection apparatus according to claim 7, wherein the display controller displays the index based on an average of a plurality of detection results of the focusing state by the focus detection unit; and wherein the display controller reduces the number of times of averaging of detection results of the focusing state in a case where the amount of change in position of the first region is equal to a first amount in accordance with the predetermined operation, compared with a case where the amount of change is equal to the second amount smaller than the first amount.
 14. The focus detection apparatus according to claim 1, wherein, in a case where the setting unit sets the first region based on information regarding the predetermined subject, the display controller displays the index based on a latest detection result of the focusing state by the focus detection unit.
 15. The focus detection apparatus according to claim 1, wherein the setting unit sets a plurality of first regions within an image region of the predetermined subject.
 16. The focus detection apparatus according to claim 15, wherein the display controller displays the index based on a focusing state detected by the focus detection unit in the plurality of first regions laid out horizontally in a case where the direction of the predetermined subject is forward or backward and displays the index based on a focusing state detected by the focus detection unit in the plurality of first regions laid out vertically in a case where the direction of the predetermined subject is sideways.
 17. The focus detection apparatus according to claim 1, further comprising: a storage unit configured to store a relative position of the first region with respect to an image region of the predetermined subject, wherein when information regarding the predetermined subject is not obtained, the setting unit sets the first region based on the relative position stored in the storage unit.
 18. The focus detection apparatus according to claim 1, wherein the display controller displays the index in a first mode in which the focusing state changes in accordance with an operation performed by a user.
 19. The focus detection apparatus according to claim 18, wherein the setting unit sets a second region within an image generated by the image pickup unit; and wherein the focus detection apparatus has a second mode in which the focusing state is changed automatically based on the focusing state detected by the focus detection unit in the second region.
 20. The focus detection apparatus according to claim 19, wherein the display controller does not display the index in the second mode.
 21. The focus detection apparatus according to claim 19, wherein the size of the second region is larger than the size of the first region.
 22. The focus detection apparatus according to claim 19, wherein the size of the second region is changed in accordance with the size of the predetermined subject.
 23. The focus detection apparatus according to claim 1, wherein the first region has a constant size.
 24. The focus detection apparatus according to claim 2, wherein the predetermined subject is a face, and the distinct region is a region corresponding to an eye.
 25. The focus detection apparatus according to claim 2, further comprising: an estimating unit configured to estimate a position of the distinct region based on information regarding the predetermined subject or a distinct region detecting unit configured to detect the distinct region from an image region of the predetermined subject.
 26. The focus detection apparatus according to claim 1, wherein the focus detection unit detects a defocus amount and a defocus direction based on a pair of image signals generated by the image pickup unit, and the index varies in accordance with the defocus amount and the defocus direction.
 27. The focus detection apparatus according to claim 1, wherein the display controller displays an indicator representing the first region along with the index.
 28. A control method for a focus detection apparatus, the method comprising: performing photoelectric conversion on a luminous flux having passed through an imaging optical system by an image pickup unit; detecting a focusing state based on a signal generated by the image pickup unit; setting a first region within an image generated by the image pickup unit; controlling such that an index representing the focusing state detected by the focus detection unit within the first region can be superimposed on the image; and obtaining information regarding a predetermined subject in the image, wherein a position of the first region is set based on information regarding at least one of a position, a size, and a direction of the predetermined subject in the image. 